Evaluate teaching quality of physical education using a hybrid multi-criteria decision-making framework

The teaching quality evaluation of physical education is an important measure to promote the professional development of physical teachers, improve the quality of school teaching and personnel training. It is helpful for students to achieve all-round development and better meet the needs of modern talents in the new era. This study aims to establish a novel MCDM (multi-criteria decision-making) framework for evaluating teaching quality of physical education. First, PFNs (picture fuzzy numbers) are suggested to reflect dissimilar attitudes or preferences of decision makers. Then, the typical SWARA (Step-wise Weight Assessment Ratio Analysis) model is modified with PFNs to calculate the weights of evaluation criteria. Considering that some criteria are non-compensatory during the evaluation process, the idea of ELECTRE (elimination and choice translating reality) is introduced to obtain the ranking results of alternatives. Specially, the MAIRCA (Multi-Attribute Ideal-Real Comparative Analysis) method is extended to construct the difference matrix in a picture fuzzy environment. Last, the hybrid MCDM model is utilized to assess the teaching quality of physical education. Its superiority is justified through comparison analyses. Results show that our approach is practicable and can provide instructions for the teaching quality evaluation of physical education.


Introduction
As a key component of school education, school physical education plays a vital role in promoting students' all-round development and cultivating students' innovative spirit and practical ability [1]. The teaching quality evaluation of physical education can not only effectively control the implementation and reform of physical education teaching, but also be used in the training, recruitment and selection of physical education teachers [2]. Scientific teaching quality evaluation system and methods are very important to improve the quality of school teaching and train sports professionals. Some researchers have focused on the teaching quality evaluation of physical education. Recently, Zeng [3] evaluated physical education teaching quality based on the data mining technology and hidden Markov model; Duan and Hou [4]  solved the teaching quality evaluation problems with the simulated annealing algorithm; Cheng [5] introduced web embedded system and virtual reality to assess teaching quality of physical education; Williams et al. [6] discussed the teach quality of physical education using psychology, a Game Sense Approach and the Aboriginal game Buroinjin. However, these methods are qualitative or applied only to certain decision-making environments. Jiang and Wang [7] established a fuzzy comprehensive evaluation model for evaluating teaching quality of public physical education; Mishra et al. [8] assessed physical education teaching quality based on intuitionistic fuzzy similarity measures; Liu [9] established an intuitionistic fuzzy TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution) model for the teaching quality evaluation of physical education. To evaluate the teaching quality of physical education more comprehensively, evaluators from many sides, such as field experts, authorities and students, are usually needed. In such a case, multiple dissimilar attitudes or perceptions may arise at the same time. Therefore, PFNs (picture fuzzy numbers) [10] are suggested to describe uncertain evaluation information. The most obvious feature of PFNs is it that it contains four different membership functions, which can accommodate diverse viewpoints and describe decision makers' ambiguity, uncertainty and inconsistency sufficiently.
To date, plenty of decision-making methods have been extended for tackling evaluation issues under picture fuzzy circumstances [11]. Some researchers established various picture fuzzy decision-making models based on dissimilar aggregation operators. For example, Wei [12] defined the arithmetic and geometric aggregation operators with picture fuzzy information; Khan et al. [13] discussed the picture fuzzy Logarithmic aggregation operators; Jana et al. [14] introduced the picture fuzzy Dombi aggregation operators; Ateş and Akay [15] extended the Bonferroni mean operators in picture fuzzy environments; Jin et al. [16] explored the weighted interaction arithmetic and geometric aggregation operators with PFNs; and Ashraf et al. [17] proposed some Maclaurin symmetric mean operators in picture fuzzy environments. However, they may lead to information loss or distortion during the aggregation process.
Liang et al. [18] integrated the EDAS (evaluation based on the distance from average solution) with ELECTRE (elimination and choice translating reality) models for evaluating cleaner production of mines; Wang et al. [19] investigated the picture fuzzy MABAC (multi-attributive border approximation area comparison) technique to solve risk assessment problems of energy contracting projects; Ashraf et al. [20] modified the TOPSIS method for solving decision-making issues in picture fuzzy surroundings; Wei et al. [21] presented a picture fuzzy bidirectional projection approach to assess safety of construction project; Luo et al. [22] selected the site of tailings pond with an extended WDBA (Weighted Distance Based Approximation) method under picture fuzzy circumstances; Gündoğdu et al. [23] assessed public transport service quality with the AHP (Analytic Hierarchy Process) and linear assignment model in picture fuzzy surroundings; Aydoğmuş et al. [24] developed a CODAS (Combinative Distance-based Assessment) algorithm to select suitable ERP (Enterprise Resource Planning) systems within picture fuzzy situations; Korucuk et al. [25] proposed a CoCoSo (combined compromise solution) method with PFNs for evaluating ideal smart network strategies for logistics companies; Gireesha et al. [26] introduced a picture fuzzy MARCOS (Measurement of Alternatives and Ranking according to Compromise Solution) method to assess could service; and Yildirim and Yıldırım [27] modified VIKOR (VlseKriterijuska Optimizacija I Komoromisno Resenje) approach with picture fuzzy information to assess the satisfaction level of citizens in municipality services.
Recently, a novel multi-criteria evaluation method, the MAIRCA (Multi-Attribute Ideal-Real Comparative Analysis), was proposed by Pamucar et al. [28]. It obtains the ranking orders of alternatives based on the gap between theoretical and real results [29]. Nevertheless, the classical MAIRCA approach with exact numbers cannot capture decision makers' ambiguity and vagueness. In this case, few scholars combined MAIRCA with fuzzy set theory to resolve decision-making issues. For instances, Chatterjee et al. [30] Evaluated the performance of suppliers with a rough MAIRCA technique for green supply chain implementation in electronics industry; Adar and Kilic Delice [31] modified the MAIRCA model with hesitant fuzzy linguistic term sets; Boral et al. [32] integrated AHP with MAIRCA in a fuzzy environment for fuzzy failure modes and effects analysis; Bagheri et al. [33] identified the uncertainty factors on the constructional project with an extended MAIRCA under hesitant fuzzy circumstances; García Mestanza and Bakhat [34] presented a fuzzy MAIRCA for evaluating over-tourism; Zhu et al. [35] established an extended MAIRCA model to assess biological inspiration for biologically inspired design in fuzzy rough environments; Ecer [36] modified the MAIRCA with intuitionistic fuzzy numbers to choose coronavirus vaccine in the age of COVID-19. However, the MAIRCA has neither been extended in a picture fuzzy environment, nor been utilized to deal with teaching quality evaluation issues.
In view of the above inspiration, the intention of this study is to investigate the physical education teaching quality evaluation problem under a picture fuzzy environment, which has strong theoretical and practical significance. The major innovation and contribution are: First, PFNs are utilized to depict complex and fuzzy evaluation information. As a result, different opinions and judgments of decision makers can be expressed with four membership functions. Second, the classical SWARA (Step-wise Weight Assessment Ratio Analysis) is extended with PFNs to compute criteria weights. Third, a new MCDM (multi-criteria decision-making) method is proposed. In this framework, the idea of MAIRCA is introduced to construct difference matrix, while the ELECTRE method is modified to obtain ranking results in picture fuzzy environment. Fourth, the hybrid MAIRCA-ELECTRE method is utilized to tackle teaching quality evaluation problems of physical education. Further, its feasibility and effectiveness are proved.
The rest is outlined as follows. Preliminaries of PFS and PFNs are described in Section 2; a new methodology, the hybrid MAIRCA-ELECTRE method, is developed with four phases in Section 3; a case of teaching quality evaluation of physical education is studied to testify the feasibility of our approach in Section 4; comparison analyses and discussions are made to verify the efficiency of the MAIRCA-ELECTRE method in Section 5; and final conclusions are drawn in Section 6. Preliminaries Definition 1. [10] Given three membership degrees a(θ), b(θ) and c(θ), then is the positive, neutral, negative and refusal membership degree, respectively.

Definition 6.
[18] If α 1 = (a 1 , b 1 , c 1 ) and α 2 = (a 2 , b 2 , c 2 ) are two arbitrary PFNs, then their Euclidean distance is dða 1 ; a 2 Þ ¼ ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffiffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi Hybrid multi-criteria decision-making framework In this section, a hybrid decision-making framework is established to deal with multi-criteria evaluation issues in picture fuzzy environments. The flowchart of this method is shown in

Phase I: Establishment of assessment matrix
In this phase, decision-making information is acquired under picture fuzzy circumstances. Afterwards, a standard assessment matrix is constructed by normalizing the original evaluation data.
When evaluation criteria belong to different types (benefit or cost criteria), they need to be normalized withã ( Consequently, a normalized decision-matrixẼ ¼ ðã ij Þ m�n can be established.

Phase II: Determination of criteria weights using modified SWARA method
In this phase, the criteria weight values are calculated using a modified SWARA method under picture fuzzy contexts.
Step 1: Get a descending order of criteria. Decision makers rank all evaluation criteria according to the significance of criteria, their knowledge, experience and preference. The most important criterion comes first, followed by the less important criterion, until a descending order of criteria C (1) Step 2: Compute the comparative importance index.

PLOS ONE
Evaluate teaching quality of physical education using a hybrid multi-criteria decision-making framework Experts are asked to estimate the comparative importance degree of criterion C (j-1) over C (j) with a PFN β j = (a j , b j , c j ) (a j � 0.5), then the comparative importance index p (j) can be computed with Step 3: Calculate the importance degree of each criterion. According to the comparative importance index, the importance degree is

Phase III: Construction of difference matrix using extended MAIRCA approach
In this phase, the idea of MAIRCA method is borrowed to construct a difference matrix.
Step 1: Obtain the picture fuzzy priority for the selection of each alternative. Generally, experts are unbiased with each alternative before making decisions. That is, they have no preference for any certain alternative. In this case, the picture fuzzy priority P A i (i = 1,2,. . ., m) for the selection of each alternative is visually equivalent, denoted as Step 2: Establish the theoretical assessment matrix. According to [28], the elements in a theoretical assessment matrix are the coefficient of the preference picture fuzzy priority P A 1 (i = 1,2,. . ., m) and the criteria weight ω j (j = 1,2,. . ., n), namely, t ij ¼ P A i � o j . Therefore, a m × n theoretical assessment matrix T can be established as follows: . .
Step 3: Establish the real assessment matrix.
Based on the initial normalized evaluation matrix and the theoretical assessment matrix, a real assessment matrix R can be established as follows: where is the maximum value of alternatives under criterion C j , andã ij is initial normalized evaluation value, denoted as a PFN.
Step 4: Construct the difference matrix. Based on the theoretical and the real assessment matrices, their gap can be formed into a difference matrix G as follows: Phase IV: Acquisition of ranking orders using ELECTRE III method In this phase, the ELECTRE III method is adopted for obtaining the alternatives' ranks.
Step 1: Calculate the concordance degree between two alternatives. The concordance degree between A i and A k (i, k = 1,2,. . ., m) can be calculated with where Cðg ij ; g kj Þ ¼ , r j and Δ j are the preference and indifference thresholds under criterion C j , and 0 � r j < j.
Step 2: Compute the credibility degree between two alternatives. The credibility degree between A i and A k (i,k = 1,2,. . .,m) can be calculated with where DI j ðA i ; Step 3: Acquire the ranks according to the ranking index. The ranking index of A i (i = 1,2,. . .,n) can be determined by As a result, the alternatives' ranks are acquired based on the values of RI(A i ). Namely, the bigger the value of RI(A i ), the better the alternative A i , and the best alternative A � has the biggest value of RI(A � ) = max(RI(A i )).

Case study
In this section, a case of teaching quality evaluation of physical education is investigated.

Background
Physical education is an indispensable part of the development of education. It aims to cultivate more excellent comprehensive talents for the country and is an important cornerstone to promote the vigorous development of various undertakings. The teaching quality evaluation criteria system of physical education is shown in Table 1 [4].

Illustration
Suppose there are four physical education colleges {A 1 , A 2 , A 3 , A 4 }, the proposed methodology in Section 4 is adopted to evaluate their teaching quality.
In Phase I, the assessment matrix is constructed in a picture fuzzy context. First, ten decision makers (including four government representatives related to physical education, four professionals and scholars in the field of physical education, and two sports practitioners) are organized to make assessments for them based on Table 1. In view of the unquantifiable nature of most criteria, PFNs are finally used to describe the evaluation results. As a result, an

PLOS ONE
Evaluate teaching quality of physical education using a hybrid multi-criteria decision-making framework initial decision-making matrix can be built, as shown in Table 2. Because five evaluation criteria are all benefit criteria, they do not need to be normalized. Namely, the normalized decision-making matrix is the same as the initial matrix. In Phase II, the criteria weights are determined. First, decision makers reach a broad agreement that teaching contents (C 2 ) should be first considered and finally the teaching link (C 4 ) when assessing the teaching quality of a physical education college. Hence, they recognize the most important criterion C 2 and the least important criterion C 4 , and repeat this process among the remaining criteria until all criteria are ranked in a descending order C 2 � C 1 � C 5 � C 3 � C 4 . Thereafter, experts evaluate the comparative importance degree between two adjacent criteria with PFNs, as shown in the second column of Table 3. For instance, if eight experts agree that teaching contents (C 2 ) is more important than teaching attitude (C 1 ) while the other two experts disagree, then the comparative importance degree β 2 = (0.8, 0, 0.2).

PLOS ONE
Evaluate teaching quality of physical education using a hybrid multi-criteria decision-making framework Based on Eq (11), a difference matrix is constructed in Table 6. For example, g 11 = t 11 -r 11 = 0.0598-0.0345 = 0.0253.
In Phase IV, the ranking orders are acquired. First, let r j = 0.01, Δ j = 0.02 and Λ j = 0.05, then the concordance degree between two physical education colleges is computed using Eq (12), as shown in Table 7. For instance, Thereafter, based on Eq (13), their credibility degree is calculated in Table 8. For example, Last, the ranking index of each physical education college is obtained with Eq (14), as follows:

Analyses and discussions
In this section, analyses and discussions are made to verify the effectiveness of our method.

Sensitivity analyses
To testify the sensitivity of the proposed method, the variations of ranking results are analyzed by changing the criteria weights in

PLOS ONE
Evaluate teaching quality of physical education using a hybrid multi-criteria decision-making framework (0.239,0.382,0.114,0.095,0.170). The ranking orders with different values of z are listed in Table 9.
Toss explore the relationship between ranking orders under different z valuesss, the Spearman's rank-correlation test [37] is adopted with the following two parameters: ℏ ¼ ƛ � ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ðm À 1Þ where ƛ 2 [-1,1], m is the number of alternatives and dis(A i ) is the difference degree of alternative A i under different ranking results. When the value of ƛ is close to 1 (or -1), it means a close correlation between two rankings. Thus, based on Eqs (15)- (16), the ƛ and ℏ values are computed, as shown in the sixth and seventh columns of Table 9. For instance, when the z value varies from 0.1 to 0.2, the ranking order changes from  Table 9, it can be seen that the best college A 1 is recognized when 0 � z < 0.6, while the ranking orders change a lot when z value varies from 0.5 to 0.6. The reason may be that with the increase of z value, the significance of teaching contents C 2 and teaching attitude C 1 is diminished. At the same time, with the increase of weights of other criteria (teaching approaches C 3 , teaching effect C 4 and teaching link C 5 ), the ranking orders are dissimilar with different z values (z � 0.6). On the contrary, when the importance of teaching contents C 2 and teaching attitude C 1 is emphasized, the ranking orders keep relatively stable and college A 1 is highlighted.

Comparison analyses
Several different decision-making approaches are applied to deal with the teaching quality evaluation of physical education. (1) Approach 1: This approach is presented by Wei [12]. He defined the weighted arithmetic average aggregation operator of PFNs and then proposed Approach 1 on the basis of this operator.
(2) Approach 2: This approach is based on the weighted geometric average aggregation operator of PFNs [12].
(3) Approach 3: This approach is proposed by Wang et al. [19]. They defined the distance of PFNs, and then extended the classical MABAC model with picture fuzzy information.
(5) Approach 5: This approach is introduced by Liang et al. [18]. They combined EDAS with ELECTRE models to get the ranking orders of alternatives.
(6) Approach 6: This approach is a classical MAIRCA approach, whose inputs are PFNs. The discrepancy between Approach 6 and our approach is the determination of ranking index of physical education colleges. That is, the ranking index of physical education colleges is computed by X n j¼1 g ij in Approach 6. The smaller the value of X n j¼1 g ij , the better the physical education college A iss .
Then, the ranking orders using the above six approaches are obtained and listed in Table 10. From Table 10, it can be seen that diverse ranking results are obtained with dissimilar approaches. To determine the optimal ranking order, a scoring mechanism is employed. That is, a higher ranking score is assigned to a better alternative. Take the ranking order A 4 � A 1 � A 2 � A 3 obtained by Approach 1 as an example, the ranking score of alternative A 4 is RS 1 (A 4 ), that of A 1 is RS 1 (A 1 ) = 3, that of A 2 is RS 1 (A 2 ) = 2, and that of A 3 is RS 1 (A 3 ) = 1. In this manner, the ranking orders with above six approaches and our method are scored in turn. As a result, the total ranking score of each alternative is computed by summation as follows: RS(A 1 ) = 3 + It is clear that the ranking order with MAIRCA-ELECTRE approach is the same with the optimal ranking order. It demonstrates that our approach is more pertinent than other six approaches in disposing physical education teaching quality evaluation issues. Then, the method proposed in [38] is adopted to measure the gap between each ranking order listed in Table 10 and the optimal ranking order. That is, their gap can be determined with the following equation: where H i is the optimal ranking order of alternative A i , and h i is the ranking order of alternative A i with an approach listed in Table 10.
Based on Eq (15), the total gap between the results of each approach and the optimal ranking order is calculated, as shown in the last column of Table 10. It is true that Approach 1 has the largest gap, followed by Approach 2. Both the gap of method 5 and the gap of method 4 are smaller than that of Approach 3. Furthermore, our approach (MAIRCA-ELECTRE) achieves the smallest gap, followed by Approach 6. Possible reasons for their differences are: Approach 1, 2, 3, 4 and 6 cannot deal with the situation where evaluation criteria are not compensatory. Additionally, Approach 1 and 2 are based on different aggregation operators of PFNs. In this case, some evaluation information may be lost or distorted during the aggregation process. Although Approach 5 can tackle non-compensatory criteria, its average solutions are calculated using aggregation operators in the difference matrix construction process with the EDAS method. In this case, it may not avoid the information loss or distortion problem. Overall, results show that the proposed MAIRCA-ELECTRE approach is feasible and the most appropriate among these approaches.

Discussions
The proposed approach combines the MAIRCA method for establishing the difference matrix, and the ELECTRE method for getting the ranking result of physical education colleges. The largest advantages of our method are two-fold. First, compared with other popular methods (such as VIKOR, MARCOS [39], DEMATEL (Decision Making Trial and Evaluation Laboratory) and FUCOM (Full Consistency Method)), the distance-based MAIRCA method is easily implemented and can reduce information loss as much as possible. On the other hand, compared with some classical approaches (such as the TOPSIS, WDBA and MABAC), the ELECTRE method can effectively handle evaluation criteria with non-compensate. Comparative analyses reveal that our approach is more suitable in settling decision-making problems with non-compensatory criteria. Furthermore, SWARA is introduced for recognizing criteria relative significance and weights subjectively [40]. Compared with other weight-calculation technologies (such as AHP and BWM (Best-Worst method)), it gives the chance for experts or decision makers to determine their priority according to the current conditions of social and economy [41]. One limitation of the model may be that all decision makers are assumed to have complete rationality and equal weights, which should be overcome by introducing other theories in the future.

Conclusions
Physical education is very important in promoting comprehensive development of students and cultivating sports specialists. This study focused on the teaching quality evaluation of physical education with a novel fuzzy MCDM support framework. First, the teaching quality evaluation indexes of physical education were identified. With respect to the intricacy of teaching quality evaluation and the diversity of human beings' opinions, PFNs were used to indicate fuzzy decision making data with four membership functions. Then, the modified SWARA model was established to determine the weight vector of evaluation criteria in picture fuzzy environments. Thereafter, the MAIRCA approach was extended with PFNs to construct the difference matrix. Furthermore, the ELECTRE III method is utilized to get the ranking orders of physical education colleges. The main advantage of such approach is that it can properly deal with non-compensatory criteria during the teaching quality evaluation process. Comparative analyses indicated that the proposed approach was workable and beneficial for the teaching quality evaluation of physical education, and could offer guidance for the performance appraisal and selection of physical education teachers. However, there are still some limitations of the proposed methodology. For example, experts are assumed to have equal weights, which may be not appropriate in some cases. In the future, some weight-calculation technologies should be explored to determine the weight of each decision maker especially in a group decision-making environment. Second, evaluators are regarded as completely rational. In the future, the proposed approach can be improved by considering the limited rationality or psychological characteristics of decision makers during the evaluation process.